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ABSTBACI . ^ . ' 

This exploratory study was designed to deteraine v 
w.hether those students enrolled in courses systeaatically des'^igned by 
faculty teaas to incorporate nontrad^tional and ihnovative 
instruction differed froa students not taking these courses in \ 
attitudes toward academic acd nonacadeaic exper'ience* Questionnaires 
were seat to a sajaple gt UOO freshman students; data are based on the 
379 replies received. Stepwise discriminant analysis indicated that ' 
two factor dimensions^ interest value and practical appeal ^ were 
rated more positively by students in systematically designed courses 
than by other students* Findings suggested that systematic 
instructional development efforts may have^ positive" impacts beyond 
the course level* Statistical data tables are included in the report^ 
and references are attached* (Author/KP) ^ . 
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ABSTRACT 



The purpose of this exploratory study w§s to determine whether 
freshman students- enrol led in two or more of seven courses which had 
been systematically designed by faculty teams differed^^p^m students 
not taking these courses in their attitudes toward their academic and 
ndn -academic experience. A stepwise discriminant analysis indicated 
that two factor dimensions, termed Interest Value and Practical Appeal, 
best distinguished between the two samples. Freshmen taking two or 
more "system^atically designed" courses rated their academic program in 
•a significafitly more positi've d-irection on both dimension^ than did ^ 
students not enrolled in these cour^ses. The findings suggest systematic 
instructional development efforts may have positive impacts beyond thje* 
' course, level . 



TheH.itera.ture on experimerttal , innovative and non-traditional ^ 
instpucticvi in higheir educa-tioi) has grown-rapidly since the mid-1960's. 

•Onemeed only glaoce at the studies rjjviewed by Trent and Cohen (1973) 
^to' appreciate this proliferation. One level of research in th*is area 

• has dealt with student responses to very specific instructional tech- 
nologies (e.g.r television and'compute.r-assisted instruction) or instruc- 
tional systems, e.g), the.Jiuto-tutorial system (Post^Jlhwalt?, Novak Bnd 
Murray, .1964)^ and the "Keller Plan" or Personalized SysJbm of Instruction 

' (kell-er," 1968). ' ' < 

For. example., Mathi%, Smith and Hansen (1970) found that students 
had generally favorable, attvltude^Jxiward computer-assisted itistruction 
b.efore experiencing if and that these^ositive attitudes remained essen- 
tially ui^cl^anged subsequentt.o being exposed to it. Davis, Johnson and 
Dietrith (1969), Deeming ff9B6) and Menne, Hannum, ICl ingensmith and -^Jord 
(1969) all report positive student attitudes toward televised lectures, 
although significant increases tn achievement are not so' clearly substan- ,* 
tiated. trent a^ Cohen (-1973) in reviewing a comprehensive study of 
multi'med^. auto-'tutorial techniques .on engineering studaots by Trent«'(r970) 
suggest aN^^end t9ward positive relationships between multimedia instruction 
and performance on media-rel^Qd exaqjination tasks^and between multimedia 
instruction and attitudes tovJard course experiences. ' ' ' ■' 

The eyaluitive, research cortducted on the Personalized System of 
Instruction. or the "Keller Plan,", is peVha()s the most extensive of all. 
Green (1^70), Riner (1972), Roth (l573)'.and Smith, Grey and McCauley (1973) ' 
allj^eport generally positive student attitudes toward >xam-tutor-ia-ls -and 
self-pacing aspecfs of the 'Personalized System q^Instruction. - Roth and 
'Sm\thA Grey and -McCauley also report higher scores. on measures of course 
achievement by students taking engineering courses under theJ'Kelier "Plan" 
than by students taking the^- same 'courses offered in a conventional format. " 
A comprehensive revie'w of research on the Pekonal iz.ed jSystem of Instruction 
has been publ i shed by Kul i K\ Kul i k and Carmichael ' ( 1 974 )». ^They coftcl ude " 
that, in terms of .both' student achievement and student' attitudes toward. 



'instrurtion, the Personalized System of Instruction is always the 
equivalent and, in many cas6s, a significant improvement over conventional 
methods. 

A second level of research in this area, has focused on. the evaluation 
of curricuUr or instructional experiments involving entire institutions 
(e.^., Gaff, 1970; Mor'gan, 1972) or major' units within an insti^tution • 
larger than a department (e.g., Siebel , 1973; Stakenas, 19.72). Aithotigh 
the results of these evaluations have been generally favorable-in terms 
of measured impact and students' attitudes toward their total educational ' 
experience, the organizational obstacles to broadly conceived instructional 
change in many institutiorts are pervasive and often prohibitive (Hefferlin, 
1969). This may be particularly true when inter-departmental cooperation' 
^'s required (Mayhew and Ford, 1971). Thus, while insti tut jon-wide program 
innovations may^haVe more extensive impacts than instructional innovations 
directed at the course level, the. former may also be substantially more^^ 
difficult to bring about than the latter. . . , 

Much of the literature cited above deals effecti\Aely with- student 
attiUides toward broadly implemented ^curricular innovations or-witih stu.dent 
responses to individual courses or units of experimental /non-t?'aditional 
instruction. Little research, however, , appears to have focused on the 
relationship between exposure to experimpntal/non-traditional instruction 
at the course level and student attitudes toward more global aspects of 
college such as Ihe quality of the academic program i» general. 

The purpose of this'study was to^determine whether freshman students 
enrolled in courses which had ^been systematically designed by faculty 
teams working with an on-campus instructional development center differed 
from students not taking these courses in ratings of their academic 
program, ratings'* of their non-academic life, amount of informal inter- 
action. With faculty and^degre'e of i.nvolvement in extracurr^icular programs. 
The importance of such research is twofold. First, ft is*aimed at deter- 
mining the extent to which systematically designed instructional efforts 
may have impact beyond the course level; and second, it e)^plores the \ 



t)Otential impact c^rived from establishing units which institution-* 
.dl ize thosef efforts, 

■% ^ " " 

^ " ■ ^METHODOLOGY * ' ■ ' 

' Sample ^, 
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The'/setting for the study was Syracuse University, a large, -private - 
university with, a total undergraduate enrol Iment -of approximately 10;0Qd 
students Located in Central Nqsjv York State:. A simple random^ample of 
500 freshmen was drawn by computer from the population of freshmen 
enrolled in the College of Arts and' Sciences at that -institution. The 
Arts and Science 'population from which the sample was drawn was apprpxi- 
jnately 54% male and 46% female, as estimated at the beginning of •the^'\ 
spring 1975 semester. * / 

Instrument * > 

■ As 'a measure of their ratings'^of their acadejnic program, students' 
were asked to rate the statement "I NAVE FOUNO MY ACADEMIC Pf^OGRAM AT 
S.U. TO BE:" on' the Adjective Ratiing^ Scale (ARS). (Kelly and Greco, 1975). 
The ARS was alao used- by^the^ student 'sajnple to respond to the statement 
"I HAVE FOUND MY NON-ACADEMtC LIF^ AT S.U. TO BE:" The, ARS consists of ^ 
twenty-four adjectives (e.g., good, enjoyable., demanding, boring, useless, 
.practical, different, interesting, dull) against which the respondent 
rates certain, specific statements using the following four-point scale: 
1 = extreme fy, 2 = very, 3 = somewhat, 4 = not at all. 'The adjectives 
initially selected in the development of the instrument were chosen 
from. descriptors typically employed by students to rate the in^ruction 
received in individual undergraduate courses.' A series of factor 
.analytic studies u^ing different methods of factor analysis indicated 
a stable underlying solution consisting^f five factors. Subsequent, 
validational analysis indicated substantial correlations' V = .58 to 
..93 in magnitude) among the five factors of the ARS and the eval.iiation\ 
potency and activity dimen'Sipns o^ the Sema-ntic Differential jKelly and^ 
Greco, 1975).^ ' 

Additional items on the instrument asked students to estimate both 

7 



3 

6 



the number of^times during the-semester they had met infornially with 
faculty jnembers for ten minutes or more and the approximate' number of 
extracurricujlar activities in* which they had participated during'the 
year. The questionnaire also asked stude'n-ts- to respond to» a number of 
other items; such'as expected major, sources oX^ersonal satisfaction ;« 
and i'nfluente, educational goals and "Clark-Trow^*^ typology which were 
a suf>plemen'tary part of the present analysis.*" The four educational 
goals* which students Vankecl in terms of their importance were^ 1) basic 
geheraV education and. appreciation, of ideas*, 2) knowledge and skill<s 
d-irectly 'applicable J^o a career, 3) increased knowledge of self, values 
*ar)d goals, and 4) the -enhancement of ^interpersonal^ skills. The CUrk- • 
Trow typology, based tfn students' orientatjons toward ideas and their ' 
identification with the institution, presented respondents' wi th- four 
statemejji*' (labeled w, x, y and'z) descrj-bing different kinds of students 
and asked them to select-the one which most closely (Ascribed themselves. 
The four statements represented Clai;-k-Trow' s "Vocational," "Collegiate," 
"Acadeiilic" and "Non-Conformist" types and were drawn from Gottlieb and 
•Hodgkins (1968). SI ight modifications were made in the original statements 
^to eliminate References to gender. 

In o'rder to separate freshmen who took systematically designed courses 
from those who did not, students were a^Bd to indicate whether they had - 
taken, or were presently enrolled in, any of .seven large undergraduate 
"courses which had been develo^d by faculty teams working irr collabera- 
'tion with an on-campus instructional developjnent unit. The development ' 
of each course required from six to twelve months- before the .rnitial - 
field testing*. This typically included a. four- to ^ht-w^ek intensive 
summer period in which the faculty team was'paid full time to work with 
a professional developer in the design and preparation of the course for 
the academic year. A more deta'iled description of the general process 
followed in the development of each course is found in Diamond , 'et. al . ■ 
(1 975). • ^ ' \ ■ . ' . - 



The^ predoitiinant ins^uctioricil aspects gf each^of the seven courses - 
1/sted briefly below. - While most the courses have distinctive 
.rugtional features whidh could be termed non-traditional, i/inpvative 
personal i*2ed, it -was felt that'the term "systematically designed'*- 
s the. most appropriate general descriptor for each. ~ ^ 

*1. Communications and Society : This large enrollment, basic com- 
munications course ^employed a variety of teaching/learning 
strategies sujch as lectures, discussion sections, private 
tutorial help and:a num'ber'of seTf-instructional materials. 
' • Additi.onal features were telephone discussions by nationally 
recognized experts and a series of* optional evening enrichment 
* activities. 

2. Drugs in Perspective : This interdi sci pi inary course -presented 
a- broad view of drugs and drug education in American culture. 
'It, utilized a wide variety of seJ f-instructional materials, 
simulations, and small-group activities, A variety of ORtiona.l 
extra credit minicourses allowed students to earn from^ 3-6 
academic creditSr ^ ' 

3. foundations of Human Behavior : Based on the ".Keller Plan," 
this vj^s a three-tredit undergraduate survey course, which 
featured self-instructional materials arid permitted students * 
to cover the course content essentially at their own pace. 
Students moved from one unit to the next only after demonstrating 
sufficient content mastery on a unit test. 

4. ^ Internationalize fations : This course offered ^students a variety 

of content options .as well as a variety of .learning activities, 
which could be used to sati^sfy the instructional vobjectives of 
the course. These included computer Sim'ulation, role playing 
games, and video taping. A series of optional minicourses allowed 
students to earn from 3-6 credits. 
- 5. Introduction to. the Study of Religion : Students in this course 
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. took a two-week introductory sequence which was followed • 
by. a wide irange Qf op_tiQns. such as "Myth," "God and Reason," 
and- "Psychological Approaches to' the Study of Religion." 
Students were ro^iuired to select three of tRe available options 
to obtain t(jfee credits and then could take additional options ^ 
or avafla-ine minicourses to earn from 1-3 %*iti6na,l credits. 
. Self-P'^ed Calculus :' Like Foundations of Human Behavior this 
course was an adaptation of the ".KeV.ler -Plan. " By using detailed 
study guides, programmed instruction sequences, and available 
totori^al he1>p, -students could cover the course material essen- 
/tially at their own pace. An.^ditional feature built into the 
/ course was f lex.ib[e.credit. Depending on how qui-cklya student 
mastered. each unit of material he or shfe could eBrn from 2-6 
credvts during the semester*. • . . > • , 

. I ntroductory^ Sociology : This was -a flexibld credit course which 
•allowed students to earn from 3-6° credits juring the semester. 
The Course presented the baste theor-ies,- concepts arid methods. of 
^ . Sociology in" a format which allowed a variety of content and 

methodology options and a choice from among alternative projects 
tp satisfy course requirements. 

Students were classified in the "systematic de^^ign" group if they' 
had taken, or. were cujrrently enrolled in, two or 'more df-the seven couf^ses. 
Those respondents wh^jidicated that they had not taken, or were not » 
presently enrolled fn, any of the seven fcourses were classified &s a - 
"conventional" group. (It should be noted that the word "conventional" 
in the preseTit stud^^is intended^on]y for classi>ication purjjoses. 
Clearly it may, not ^be thejnost appropriate term for all the courses to ' 
which students in the conventional group h^ave been exposed^) 

Response ; , , - 

The questionnaire was distributee^ by mail" to the entire sample in late 
March of 1975. Subsequent to a'^ail follow-up conducted on a random sample 
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of non-respondents approximately three weeks after the initial mailing, 
usable responses were obtained from 379, subjects , yielding a response 
rate of . 75,8%'. - The representativeness of t^e sample was indicated by* two 
factors: the high, rate of response to the questionnaire, and a' chi- 
•square analysis indicating non-significant differences between the dis- 
tribution oV responding males* and fema*les and the distribution of male^ 
and females in the population, * _ , • ' 

Forty-six Respondents intftcated that they had , taken *two -or more of 
the specified courses and thus constituted the systematic design group. 
One-hundred twenty-seven respondents had not taken any of the seven courses. 
These individuals formed thSyConventional group. From this latter group, 
46 subjects were randomly selected -to give equal N*s in both comparison 
groups and to permit later use of the remaining 8^ subjects , in the 

' conventional group for cro'ss-validation. piy^poses. 

To check for representativeness,' a secorfd independent random sample 
of 46 subjects was chosen from the conventional group and'^compared to the 
first -sample from the conventional group all variables used in -later 
analysis. Each of the mean diffe^nces noted had significance levels 
greater than .20 and thus tould be reasonably attributed to chance; 
Furthermore, a'chi-square analysis could not rejert the null hypothesis of 
chance differences between 'the di^rfepHbution- of males and females in the 
total conventional group of 127 and the distribution of males and females 

^ f n the initial siibsample of 46 from the* conventional group. 

^ The observed differences between the systematic design and conventional* 

A t 

sample groups on 1) the distribution of respondents by selTT" expected major* 
and ClarK-Trow typology choice; 2) the rarik-orderihg of the four educa- 
tional gpal statements; and 3) the means of .available Scholastic Aptitucfe 
Te^t scores were t-esfed for significance. (It should be noted that in the . 
latter analysis SAT scores were available for'32 respondents in the system- 
atic design group and 35 respondents, in the conventional grou|). Jhe chi- 
square values obtained for the distribution of respondents by sex , expe c ted 
major and Clafk^Trow typology choice wer*e; 0.39 ^(df =1), 4.48 (df = 4) ' 
and 3.88 (df = 3), respectively. None of^the three chi-sguare values was' 




significant at p <• .50. Sinn" larly,' the Mann-Whitney test for the equality 
,of means in rank-ordered data .(Siegel , 1956) indicated non-significant 
differences between the two groups in the mean ranking of all four 
educational goal st^tements.s Filially, .t'-tests indicated non-significant 
differences between, the systematic design and conventional' groups on 
the^Verte'l* and Quanti'tati ve scores of the SAT. The means and-.standard 
deviatio ns for th e Verbal^mre were: systematic design; x = 512, - 
= 99.77; converitlonal: »x = 520, S.D. = 10l;89; (t = 0.3^). The' 
correspondtng values for the Quantitative score were systematic design:' 
X. = b44, S.D. = 104.6;'convent1ondl : x '^ 539, S.D. = 92*51 ; (t = 0.20). 
As the* above -set of analj^ses indicates, the two sample groups were 
essential ly* homogeneous on all variables tested. 

\Statistical Analysis^ * ^ . . 

AUhough the factor strocture of the Adjective Rating Scale was 
-previously developed on a sample of 769 subjects, the stimulus statement.' 
to which the subjects responded pertained to specific courses (Kelly^and 
Greco,^975). In the present study students jvere be'ing asked to rate 
somewhat 'broader experiences, i.e., the academic program and their non- 
academic life. It was, therefore, judged necessary to empirically deter- 
mine the^factor structure which held for this somewhat different use of 
the ARS and verify the -degree of structural. similarity with the original 
solution reported by KeUy and Greco,- (1975): , ; \ ' 

Analysis of the data thus^-b^gan with a principal components analysis' 
of subjects.' ARS responses. A separate Analysis was done for'each ,of • the ^ 
two statements~fate^: Following^ Kaiser's (1959) varimax criterion, 
components with eigenvalues > 1.0 were extracted and subjected^ to varimax , 
rotation to orthogonality. The rotated component? will hereafter be 
referred to as factors. "Programl^elate" (Veldman, 1967) was used to 
compare the structura^l simi/larity of the original solQtion reported by 
Kelly and Greco (1975) and the factor solution yielded by the use of the 
ARS in the pr^serit study. ^ "Program Relate" permits the comparison of factor 
strvictures from two independent sample groups hy, hoVding one structure 
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fixed and rotating the second structure on jt until maximal similarity 
is achieved among the individual test vectors^(±est'Vectors in the ' ^ 
present study are the 24 adjective scales).^ The degree of rotation 
required to achieve giaximal similarity is expressed as a matrix of- 
cosines^ which m^y be regarded as a matrix of correlations between the 
two sets of factor vectors.^ 

' 'M^an factor scales were .computed' for each respondent bj^summing his 
raw scored on variablesVwith rotated factor loadings of .40 and' above ort • ^ 
a particular, factor, and cjividing by the-number of variables, .Where'a 
variable loaded above .40 on two dimensio^6ji t was included n'n-the 
computation of factor scales for that factor orr which it had the higher , 
loading. The purpose of computing ^factor, scales by u§-ing characteristic- * 
variables rather th^ri a complete estimation method in which all variable's,-, 
regardless of their factor loadings, are used was tcr increase the internal 
consistency (alpha) reliability of the ndividual factor scales (Armor, 
197^4). At the same time, using only'those variables with high loadings 
to compute factor s<:ales may r^esult in the lx)ss of orthogonality and, lead 
to substantial inter-scale correlations. The authoVs jutiged .that it' ' 
would be preferable to optimize the internal con^'stency relicibility of 
each scale despite^the potential 'los^ of orthogonality since the latter- 
situation can be dealt with effectively by employing njferl tivariate proc^^es, 
specifically discriminant analysis. ^ \ 

The factor scaled derived froni respondents ' 'ratiflgs pf their. academic 
prpgram and their non-academic life were combined with their ntmiber bf 
informal interactiens with faculty and their participation iri extracurricular 
activities. Th^se variables forme^*the basis ^f a two-group discriminant 
function analysis (Cdol'ey and Lohnes, 1971) jto determine .the effectiveness - 
with which they ^separated the systematic design from the conventional group* 
The" academic and non-academic variables were employed as pfedictor^ ' 
variables andventered 'into the discriminant'ana lysis' in a stepwise fashion^ 
The 'crJ^,te£ijD/i. for controlling- the stepwise Selection of variables for 
inclusion in the analysis was the minin^Szation of Wilk'is Vambda. The . 
minimum F-ratio'to enter the analysi s- was ' set at 1.0./ Subsequent to . 
-dis<:riminaTit, analysis^, a classif icaticJn analysis •ba'sed on the pooled covariance 



."matrix and individual discriminant .scores was used to assess the, efficacy of 
the discriminant function obta,ined. In order. to cross-validate the ' 
Qiscnminant analysis, cl assi ficat-ton w^js performed both for" the 92 
subjects on whose scores' the discriminant function was derived, and for 
•the remaining 8?l 'sub'jects from the convention^il sample^whose scores ' ' 

■ were not included in the computation of tfie discriminant fuTiction. 

.Computer programs employed in the analysis' were "Subpregram Factor"" 
'and "Subprogram^ DiS/Crimi<iant" >rom the Statistical Package for the , '• '* 
Social Sciences , Second Edition (NIE,.et al» 1975),- and "Program Relate". 
(Veldman,' 1%7). - 

■ . , RESULTS.. 

Factor analysis of students' ARS ratings of their academic program 
and their ARS ratings of their non-r'academic life ytelded five factors 
and four 'factors respectively With eigenvalues > 1.0. The composition 
of these two sets pf factors is shown in Tables 1 and 2. Asterisked 
loadings indicate those variables on .each factor used to compute factor 
scales. Each :f actor/ has been given a tentative name'which was felt to 
represent the. un«^rlying psychological construct tapped. The reader is 
cautioned, however, against attributing surplus meaning 'to the 'factors . 
beyoQd .the scale^, which characterize them. . 

Tables 1 and 2 also s^iow the alpha or internal .consistency realibility- 
coefficients coippu ted for. each set of factor scale^. As shown in fatle 1,' I 
scales. for Factor V, Uniqueness, had a computed intei*nal consistency 
reliability of only .274. This di^nension was therefore not included in 
further analysis. Similarly, Factor IV, unnamed in Table 2, was not 
included in further analysis -because it was. judged to be uninterpretable 
within , the context of the .s1>atement rated. 

The results of '"Program'Relate" indicated a high degr«e(of structural 
similar*ity between^the original Kelly and Greco (1975) factor solution 
and the two §qlutions yielded in the present study. Cosines between the ' 
original ARS factors, and those derived from; the present samples^ ARS / 



' • -TABLE.! » * 

• • . • * ■ . - • • 

VARIMAX FACTOR LOADINGS DERIVED FROM SUBJECTS' ADJECTIVE RATING 
SCALE RESPONSES TO THE STATEMENT "I HAVE. FOUND MY ACADEMIC 
PROGRAM TO BE:" (N=379) = ' 
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-.060 


-.205 J 


.632 


NECESSARY 


5 159 


-.084 


. .739* 


.105 


.145 


.610 


PRACTICAL 


.352 


-.179 


.602* 


.015 


• -.076 • 


.524 • 


VALUABLE 


.512 


-.281 • 


.583* 


.'148 . 


-.067 


.707 


WORTHWHILE 


. .498 


. -.374 


.513* 


. .068 


, -.053 


.658 


RELEVANT . 


,322 


-.398 


.442* -• 


.124 


-.135 - 


.491 


DEMANDING 


.094 


-.024 


.069 , 


*-855* 


.125 


• .761 


DIFFICULT 


.054 


."111 


-.025 


.85^* 


^.027 


.743 


.CHALLENGING 


.318 


-.218 


.267 


. .687*. 


• . .137 


.711 . 


GENERAL 


-.025 


.386 


.011, • 


" -.078 


-.695*, 


.640 


DIFFERENT 


.353 


— .162' 


.202 > 


.154 


.549* 


,'.518 


EIGENVALUES 


9.229 


2.100 


1.527 


1.070' 


1.005 




(pre- rotated) 












• * 


EIGEfWALUES , 


5.534 


- 3.374 


2,650 


r 2.23*3 


°l\l23 . 




(rotated) 














% VARIANCE 


23.08 


J4.06 


11.01. 


9^33. ' 


* 4.67 




CUM. VARIANCE 


23.08 > 


•'%7.14 * ■ 


48.15- 


57.48 


.62.15 ■ 




.ALPHA 














RELIABILITY 


-0.898. 


0.852" 


0.817.* 


0.778 


0.274 


i 



JOTE: 1. VARIANCE PERCENTAGES ARE ROTATED FIGURES. 
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2. ASTERISKED LOADINGS INDICATE VARIABLES USED TO COMPUTE / 
• FACTOR" SCALES' AND ALPHA RELIABILITY COEFFICIENTS JOR EACH/ 
SCALE. . • • ■ / 
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TABLE 2 



VARIMAX FACTOR LOADINGS DERIVED FROM SUBJECTS' ADJECTIVE RATING 
SCALE 'RESPONS-ES TO THE STATEMENT "I HAVE FOUND MY NON-ACADEMIC 
LIFE' TO-BE:" (N=379) 



•I 



VARIABLE 



INTEREST 
VALUE ' 

— • — -r- 

.836* 

.814* ■ 

.783* 
■ .717* 

.709* 
\706*' 

.666* 
-.633* 

.605* ■ 
-.601* 

,585*' 

.565* 

.128 

.215 
-.279 
..294 - 
-.238 
'-.268 
-.275 

.375 

.264 

.391 

.353 

.029 



D^IAND/ , 
CHALLENGE 



PRACtlCAl 
APPEAL 



IV,. 
UNNAMED 



EXCITING ■• 
ENJOYABLE ' 
600D 

INTERESTING 
STIMULATING 
REWARDING 
ENLIGHTENING . 
' BOeiNG 
WORTHWHILE 
DULL 

•Valuable , 

provocatfve 

demanding 

challenging* 

difficult 

different 

irrelevant 

USELESS 

A WASTE 

RELEVANT 

PRACTICAL 

INFORMATIVE 

NECESSARY 

GENERAL 



.146 


■ .154- - 


.001 


.745 


-.052 


. .264 


-.030 


',.735 


.043 


, - .311 


-.083 ' 


■ . .718 


.073 


.318 • • 


. -.004 


.621 


.141- 


• .379 


-.049 


.668 


»213 


• .345 


.171 ■ 


.691 


••168. ■ 


.'290' 


.139 


.576 


.173 


-.194 


.319 


.571' 


.179 


.531 


.074. ' 


.685 


.097 


-.329 . 


.373 


.619 


.las. 


.556 


.086 


.694 


V .207 


.240 . 


.135 


.438 


.779* 


.088 


-.128 


• .648 


.745* 


.181 


-.020 


.635 


.735* - 


-.108 


• .106 •, 


.641 


.418* 


■ .149 * 


-.060 . ■ 


.287 


.037. ■ , • 


-.72'4*: 


.237 


..638 


-.003 


■ ^.713* 


■ .300 


.670 


.002 


-;696* 


.279 . 


> .639 


.122 


.628* 


.235 


.604 


.167 ' 


.544* 


.209 • : 


.438 


• .231 


.544* > 


.290. ° 


.586 


.211 . 


.487* 


.213' 


-45a 


-.133 


-.015 


V698 


.507 


2.113 


1.278 • 


• 1.147 




2.311 


. 4.248 : 


1 .'299 ' 





EIGENVALUES- 9.969 

(pre-rotated) 

EIGENVALUES ' 6.645 ' 
(rotated) 

'% VARIANCE • -27.66 
,CUM. VARIANCE ■ 27.66 

ALPHA RELIABILITY' 0.941 



9; 60 
37.20 
0.694 



17.70 
54.90 
0.83'6 



5.45. 

60;'35 



NOTE: 1,. , VARIANCE PERCENTAGES ARE ROTATED FIGURES. 
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ASTERISKED LOADINGS INDICATE VARIABLES USED TO COMPUTE 
FACTOR SCALES AND ALPHA RELIABILITY COEFFICIENTS FOR 
EACH SCALE. ' . ' • • 



12 



ERIC 



^5 



ratings of their academic .program ranged,,form .87 /to .98. Similar 
congruence was indicated between the 'original factors and students* AR5* 
ratings of their non-academic' life. The cosines /ranged fron\ .70 to .95. 

The matrix of intercorrelations among the niae predictor variables 
is shown in Table 3. Table 4'displays the means! standard deviations and* • 
univariate F-ra|ios for each of the predig.tor va,riables. Significant 
univariate- F-ratios were found .on two factors from students*' .'ARS ratings 
of the academic program-,- Interest Value anci Practical Appeal. The 
systematic design group* rated the academic ^program in a significantly 
more positive direction on both dimensions than did the corfventional 
course group (recall the ARS is scored 1 = extremely, 2 vejjy, 3 = some- 
what, 4 = not at all). 

The results of the stepwise discriijiinant analysis are shown in Table 
5. As indicated, 5 variables entered the analysis with an F-ratio fo 
enter > 1.0. The discriminant function based an those 5 variables 
yielded a chi-square value of 13.79, significant at the .025 level. - 
Inspection of the standardized discriminant function coefficients indicates^ 
thfiit three factors derived from students' ARS ratings of the academic 
program contributed most to the discrimination between the systematic design 
and -conventional groups. The amount ot i-nformall interaction with faculty 
and the^ Practical Appeal factor from students' ARS ratings of their non- 
academic life appeared to contribute less to the discrimination than the 
three academic variables. As indicated by the change in Rao's V, 
however,, only one of the five variables in the ^nalysisv Interest Value 
(academic program), made a significant increase in the discrimination ^ 
between the two groups. 

!The fact that 1:he significant .univariate \differei\^ce between the 
systematic design and conventional groups on Practical \Appeal (aceydemic 
program) was not reflected by a significant change in Rao's V statistic 
When Practical Appeal entered the stepwise analysis appears to be explained 
by the substantial correlation between that variame and interest Value 
(academic program), r = .73. Because of the high OprrelatT\pn between the 
twa variables and because Interest Value Jiad already, enterea the an^ llysis ,— 
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TABLE 3 



PEARSON PRODUCT MOMENT CORRELATION COEFFICIENTS AMONG NINE DISCRIMINATING 
VARIABLES (N=92) 





VARIABLE 


1 






A 


D 


0 


1 ' * o n 


1. 


INTEREST VALUE (ACAD. PROG.) 


1.00 


< 


- 








i 

• \ » * 


2. 


DULLNESS/APATHY (ACAD. PROG.), 


-.55 


1.00 




• 








3., 


.PRACTICAL APPEAL (ACAD. PROG.) 


* .73 - 


-.54 


1.00 








/ 


4^ 


;)IFFICULTY/CHALLENGE (ACAD. PROG.)- " 


.37- 


-.24 


■ .46 ■ 


1.00 










INTEREST VALUE, (NON-ACAD.. LIFE) 


.33 

* 


-.30 


' -.27 


.30 ' 


. 1 .00 


* * 






DIFFICULTY/CHALLENGE (NON-ACAD. LIFE) 


.06 


-.03 


.16 


.19 


.34 


.1.00- 




•• \ 1. 


PRACTICAL APPEAL (NON-ACAD. LIFE) 


.35 


-.28 


.37 


.31 


"..74 


^ .2r 


1,00 


; 8. 


INFOi^MAL INTERACTION WITH FACULTY 


-.32 


.T9 


'",'-.23 


-.20 


.-.17 


:-.08 


-.13 1.00 




EXTRACURRICULAR ACTIVITIES 


'-.20 


-.05 


. -.18 


-.04. 


-.02^ 


.05 


) -.16 .32 1.00 



Note: Because the Adjective Rating Scale is scored 1 =.extremely, 1 = very,' 3 = somewhat, 4 = not at all 
caution should be observed in interpreting the sigps of the cofVelatiohs between the ARS scales on 

• the one hand and INF0RI1AL INTERACTION WITH FACULTY and EXTRAGURRICULAR^ACTJVITIES on the other. 

For example, the r of ■•/32 between INTEREST VALUE (ACAD. .PROS.) land INFORMAL INTERACTION WITH FACULTY 
suggests that as the nafnber of informal interactions with faculty increases student ratings on 

, INTEREST VALUE become, (fore positive. . 



TABLE 4^ 

^IIEANS, STANDARD DEVIATIONS AND UNIVARIATE F rXtIOS FOR NINE DISCRIMINATING .VARIABLES 



VARIABLE 



SYSTiWATIC DESIGN(N=46) 
MEAN ' t^. 



CONVENTIONAL (N=46) 
MEAN .S.D. 



RATIO"" 



• INTEREST VALUE (ACAD . PROG '. ) 
DULLNESS/APATHY .(ACAD, PROG.) 
PRACTICAL APPEAL (ACAQt PROG.)" 
DIFFICULTY/CHALIENGE (ACAD. PRO'G.) 
'INTEREST VAlIiE (NON-ACAd! LIFE) -~ ' 
DIFFICULTY/CHALLENGE (NON-ACAD. LIFE) 
PRACTICAL APPEAL (NON-ACAD. LIFE) 
INFORMAL INTERACTION WITH FACULTY 
EXTRACURRICULAR ACTIVIf^ES 



2.53 ; 
3 .33 

K 

2.38 

2.42; • 

2.06 
2.85 
1.80 
3.46 
3.87 



.560 
'.•501 
.565 

.552 
.586 
- .480 
4.14 . 
7.59 ' 



2.81 
3.34 
2.64 
^2:52 
2.07 
2.91 
1.76 
4.02 
2.17 



-.469 
.642 
.573 
-.546 
.459 
7.66 
3.67 



7.05** 
43 0.04 
• 6.00*. 

0.59 

0.01 

0.24 

0.14 

0.19 " 

1.86 . 



'DEGREES OF FREEDOM = 1 AND 90 



< .025 



**p < .'01 
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, . ' TABLE 5 • . , . . . • ■ 

STEPWISE SELECTION OF^ VARIABLES FOR DISCRIMINANT ANALYSIS "(MINIMUM F TO ENTER SET AT 1.0)^ 









^—^ 1::^ _ 

/ 






STANDARDIZED „ - 






• * 




/' ■ ' 
WIIK'^S 


APPROXIMATE F 


DISCRIMINANT 








F TO ' ' CHANGE IN 


FOR TEST ^ 


FUNCTION ■ - 


•STEP 


' VARIABLE 




ENTER RAO'S V^ 


LAMBDA- 

. / ■ 


OF LAMBDA^ 


COEFFICIENT ^ 

^ 



1.. INTEREST VALUe' (ACAD. PROG.)- ■ 7.05 

2. DULLNESS/APATHY (ACAD. PROG,) ' 2.99 

3. INFORMAL INTERACTION WITH 

FACULTY 

4. PRACTICAL APPEAL (ACAD. PROG.). 1.30 

5. PRACTICAL APPEAL (NON-ACAD. 

LIFE) 

(PROGRAM TERMINATED DUE 
TO F TO ENTER <.1.0) 
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7.05** 
5.09** 

3.94* 

3.29* 

2.94* 



.919 ■ 
.638 ' 

.368 . 

.605 

.390 



TENTROID FOR SYSTEMATIC DESIGN GROUPy^ -.432; CENTROID FOR CONVENTIONAL GROUP =,.432 
QHI SQUARE FOR THE .DISCRIMINANT FUNCTION = 13.79 WITH 5 DEGREES OF FREEDOM (p < .025) 

^INDICATES THE INCREASE IN DISGf^MINATION ATTRIBUTABLE TO THAT VARIABLE 
V / 

'DEGREES OF FREEDOM RANG^ROM \ AND >90 ON STEP 1 , JO 5 AND 86 ON STEP 5. 
*p < .025 / **p < .01 
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thejnclusion of Practical Appeal did not lead to a significant 
incremental increase in discrimination between the two gcoups. i 

The results of the classification analysis are displayed 'in . 
Table 6. As the table indicates, 69.56% of the total sample of 92 
subjects^ on whose scores, the discriminant function was derived, were 
correctly classified. The overai;, correct classification, including 4 

■ the cross-validation co,nventional group,''was 63.6%. This represented a 
2,7. 2% -improvement over chance. • • 

A further substanti,a|/on of the' results is indicated by comparing 
percentages of respondents in each group who ranked their "academic work" 
first or secohd from,.^ choice of six possible areas of campus life as a 
source of personal satisfaction during their freshman year. In the • ■ 
systematic design group 63% ef the students- ranked their "academic work"' ' 

• either first or second wi.th 23..9% ranking it first and 39.1% ranking.it 
second. This /compared .with 43.5% ih the conventional group who ranked ' 
their "academic work" either first or second as a ^source of personal 
satisfaction with only 9.7% ranking, it first and 38.3% raking it second^ 
A Mann-Whitn^y Test was carried out for- the rankings of the two groups 
on this item. The mean rank for the systematic design group was -2.26 

'While the mean for the conventional group wa^'2.73. A 2 value of 2.03 

.was obtained, significant at p < .05. 

Additional Analysis 

In order to determine pos^ibl^^fferences between the systematic 
design and conventional groiips on personality variables and initial « 
expectations of the col leg/ environment, a post-hoc analysis was conducted 
using the Activities Ind^-AI, a measure {)f personality needs,and the 
College Characteristic/ Index-CCI, a jtieasure of perceived environmental 
press. Both instrumeiTts are administered to all incoming freshmen 

shortly before ^np^l on campus. Thus, students' responses on the 
College Characteristics Index may be regarded as their expectations of - 
-the institution's environment. A separate stepwise discriminant analysis 

'>Was conducted^ on. -Iihe available AI and CCI 'scale scores of the systematic 
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.- ' TABLE 6 • . . 
PREDICTED CLASS ipj CATION OF "SYSTEMATIC »DESI6N"^ AND. "CONVENTIONAL" FR-ESHMEN 



ACTUAL GROUP 



PREDICTED 
SYSTEMATIC DESIGN 



"PREDICTED 
CONVENTIONAL 



% CORRECT 
CLASSIFICATION 



SYSTEMATIC DESIGN 
(N=45.) 



34 
(73.9%) 



12 
(26.1%) 



73,^% 



•CONVENTIONAL 
(N=46) 



. 16 
(34,8%) 



3a 

(55^2%) 



65.2% 



CROSS-VALIDATION 
CONVENTIONAL 

(N=81 ) \ 



35 - 

('43:2%) 



46 
(56.8%) 



' NOTE: CCKRECT> CLASSIFICATION FOR' 92 SUBJECTS ON WHOSE SCORES THE DISCRIMINANT FUNCTION 
WAS WMPUTP \ ' " I • " .• 
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OVgRALL CORRECT^ CLASSIFICATION OF 173 SUBJECTS ->^&^% 



I, 



. 1 




56.8% 



dosi(jrt and •conventional samples. Data was available for 39 of the 
systematic design subjects and 38 of' the conventional subjects. Un 
^neither analysis was the discriminant function significant afp - ^05 
(for the AI comparison chi -square = 6,89 with 3 degrees of freedom, 
p^ > .07;- for the CCI comparison chi-square = 7.04 with?4 deqrees of * 
freedom, p > .-10). Moreover, no univariate F-ratios ini;^either analyses 
reached significance at p < .10. ^ * " ^ 

" . The students initially participating in the study were followed up 
during. fhe fall 1975 semester to^^determine how membership in the systematic 
design or , conventional group was related to attrition from the institution. 
Thirty-one '(24.4%) of the 127 subjects in the^conventional group were - 
found to hav.a left the institution. Thts cdmpBred to Six (13.0%) of the 
46 subjectsr>.in the systemic design group. A one-tailed test for the 

* c 

hypothesis of a lower perter\tage of leaver^ i^n the systematic design- group 
had a significance level .10 > p > .05. Thus-, while the observed • 
difference was 'in the hypT^fehesized direction, evidence for the reliability 
of the difference was not particularly strong. • 

CONCLUSIONS AND DISCUSSION , * 

The findings of this study suggest that students enrolled during t-heir 

freshman year in two or more systematically designed courses tend to havjs 

significantly more ppsitive attitudes toward their overall .academic program 

on two dinpensionf"'^|€^rmed iwtere^^Jialue and Practical Appeal)' than do 

freshmen not enrollied ih these^ourses . Because of the substantial 

correlation between these two dimensions, however, only Interest Value made 

a sijignificant increase in the discrijnination between the systematic design 

and conventional groups when the variables were entered^ fn a stepwise 

discriminant analysis. Variables other than students' Adj[ective •Rating 

Scale views of the academic program failed' to distinguish significantly 

between the two* sample groups. ' ' * ^ ' - 

A review of the variiibles loading high on the Intei^est Value factor of 

^• 

the ARS suggests , that this dimension has both cognitive and affective - 
components. This conclusion is prompted "by the high loadings on.such 'cogni- 
tive-related adjectives as Enlightening, Interesting, and Informative. .A«- 
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affective dimension in the Interest Value factor is indicated by the high 
. loadings for such adjectives as Enjoyable, Exciting, and Stimu^lating. The 
structure of this factor and the more positive ratirrgs of students., who 
took-two or more systematically designed courses, stroagly suggest that 
■these courses'have not only a greater attraction for students but a.ls9 ^ " 
, that the attraction is broadly based in terms of the intellectual and 
emotional make-up of students: 

0 Students in the systematic design grqup also rated their academic 
* program more positively than did' students|in the conventional group along 
the dimension labeled, here. Practical Api^eal, a factor which correlated 
.73 with the Interest Value factor. PracU^ical Appeal "loaded high on the 
following adjectives-Negessary, Practical!, and Valuable--and to a lesser 
degree on Worthwhile and Relevant. One might speculate that this factor 
reflects the increased interest among students in the utility of an 
academic program for securing gainful employment following one's college 
^career. But given the high corcelation between the Practical Appeal and 
Interest Value factors, and given the highei^ scores on these two- factors 
among students enrolled in two or more systepiatical ly designed courses 
when compared with students enrolled in conventional courses, it seems 
reasonable to conclude that .systematical ly di,signed courses may lead to an 
academic program's greater student appeal in We significant areas: 
cognitive attraction, affective appeal, and perceived'utility. 

Exactly how such courses'might *lead students to^e more favorably- 
disposed toward their freshman academic experience than conventional' courses 
is more difficult to explain. As indicated earlier, the seven systematically 
'designed courses used as" a basis for grouping ih this study are widely 
varied in' their design and content. ,The most eiident— and indisputable— . 
commonality of the seven courses is that they were all developed by teams* 
of faculty members .working closely wfth an instructional de^'elopment agency 
on the Syracuse University campus. The 'frrocess, |;ulmin*ating in these . 
seven courses, affords faculty members the, time, ^professional assistante 
and financial' support necessary to effect a rigorous/ re-evaluation of 
educational and instructional philosophies, course ^content and instructional 
style. It is quite possible that the cumulative effect of this type of 
support can be associated with measurable differences i,n broad based 
instructional or .attitudinal outcomes for students. - But wljile such^a 



. ' • * 

result may be intuitively pjausible, it cannot be substantiated on the 
basis of the research, reported here. ♦ 

JhjB^iresuUs of the^ study are supportive of ipstttutional efforts in 



/ 



; the arda of systematic instructional and curricular development and indeed 
suggest that such efforts m^y have positive impacts beyond the^course level. 
However^, the ex post facto nature of survey research makes the attribution 
of resu'Ks difficult because of the myriad of student, faculty and contextual ^ 
variables which may interact to influence instructional quality and outcomes. 
Clearly a number of alternative hypotheses may be advanced to explain the 
study results. . ' ^ 

One alternative explanation may be the- presence of the "Hawthorne 
Effect." For a substantial number of freshmen the structure of many of - 

' the sj^stematically designed courses may have been sufficiently different 
from the kinds of instruction typ.ically received that they perceive them- 

^selves to be in an' experimental situation and therefore work harder and 
.find the course more intellectually and personally stimulating. 

Similarly the "Hawthorne Effect" ,may have held to some degree for 
faculty. Having invested substantial amounts of time and effort in the 
instructional deve^opmen^t process, the faculty members involved may have 

^(^l.t^ naturally developed str^g personal and professional needs to see 
the ^enterprise succeed. The ways in which^his, might positively influence 
the. quality of-t^he individual facuTty. members 's own teaching are not 
precise1y%known. . However, th€7fact that the instructional development 
process frequently jnvolves faculty with an intensive analysis of the 
assumptions they hold about the ^^Vocture of teaching and learning mfght 
conceivably reinforce an increased sensi tivity.„to the quali^ty and effective- 
ness of their own im-class teaching behaviors. Thus, one outcome of the 
substantial time and effort spent by Ta.cuVty in the systematic design of 
instruction may be a greater concern for. their owrTeffectiveness in the 
classroom, particularly in that course where they have so much invested- 
professionally. " - 

• • Related to th?is consid^ratnon^is the possibility of a self-selection 
process in termsi of the characteristics of those faculty having a high' * 
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interest in systematically developing non-traditional .instruction. As 
a group, such faci>lty.may have not only a gej;iyine^oncern for the quality 
.of their teaching, but may also, partially as a resuTt of this concern, 
represent some of ^ the institutiiDn's most effective and provocative ^ 
,|^teachers. ^ Thus in the* present study the systematic design group may 
have responded more , favorably to the academic program than their class- 
mates in the conventional group, not so much because of the particular 
instructional design of the courses in which they were enrol led, but 
•rather because their enrollment in these specific courses involved a 
greater probability of efxposure tp individually good teachers* 

Perhaps the most valid ^explanation is one which posits the potential 
interaction between course instructional .design and effective teaching. 
Work with an on-campus, agency to design courses systematically may tend 
not only ^ to draw faculty who are good teachers t6 begin with, but also 
provide these individuals with the facilities and resources necessary to 
take optimum advantage of their particular pedagogical talents. It"* 
seems entirely p'ossible that students in these courses may be re'sponding 
to an instructional gestalt in which the course design ^and instructional 
format 'amplify the faculty member's most effective teaching behaviors. 

Beyond differences in- faculty characteristics, however, the observed 
differences in the- findings might also be the result of significant 
variations in student ^aracteristics. Although the systematic design 
and conventional student groups^ appear quite homogeneous in terms of suth 
variables as sex distribution, 'expected major, orientation toward college," 
educational -goals and academic aptitude the fact that students b/ and 
large "self-selected". themselves into these groups rather than being 
randomly assigned makes it at least possible that other variables such as 
the students'cognitive style may have accounted for a significant portion 
of the observed sample differences. Research by Gaff (T970) and^ Morstain 
(1974) has indicated that students' preferring experimental or independently 
designed acSdemic experiences tend to differ from their classmates^ in 
personal ity characteristics and educational attitudes and^ orientations. 
Mjiile the results of the present study differ somewhat fro(n the findings 

22 . n 

29 • 



of 6aff^and Morstain, quite possibly unmeasured personal orientations* 

or cognitive styles which lead certain students to be attracted to the 

non-traditional features of systematically designed coursesmay also 

tend to make them more open, responsive and favorably disposed to the ^ 

• * 

instruction received in all'their courses. \ 

The study is limited in the degree to which the relationship bfetween 
attitudes toward i,hstruction and actual student behaviors is left unexplained, 
How closely the observed, statistically significant differences between the 
sample groups irj/thjis. study can be related to observable variance irr such 
student behaviors a$ academic achievement, expressed satisfaction with • 

i ' * . * 

college, and attrition is not clearly or reliably substantiated. Despite 
this limitation, hoWeyer, evidence does exist to suggest that the attitudes' 
toward instruction developed during the freshman year are critical 4n 
providing a foundation .for the student^' s subsequent openness to the. impacts 
of college (e.g., Wal lace, -1966^ Katz and associates, 1968). Indeed, as 
suggested by Kauffman and associates (1968, pp. 11-12): 

the freshman experience is .' . critical to tjie college 
and the student.' It is a time when th^ studen't's critical 
^ attitudes toward his stiUdies an,d college in general is 

formed, when the college mi>6t demonstrate the relevancy 
of 1 iberal' learning to a ready-to-bel i eve but not-yet- 
convincpd student audience*. 

Thus, while students* ratings of their academic program may. not be causally 
related to a number of readily observable student behaviors, their real 
significance may'be^in the groundwork they lay for future college impacts. 

The linkage v^ich the study tentatively identifies between c'ourse 
design and students' broadqr perceptions of their academic ^program has 
several clear and significant implications both for. the area of instruc- 
tional development and for research on the*impact of college on students^ 
The results suggest that systematically designed courses d£ make a* v 
difference;^ however, ferre]ting-out the most significant elements in such ' 
courses may require investigation of a more experimental ^nlfture. Indeed, 
suggested, by Caoipbel 1 and Stanley (1963), one implicit purpose of this 
ex post factQ and essentially correlational study was to explore whether/ 
such subseqCient^experimental investigation might be warranted. 
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The nature and extent of the interaction between the instructional 
development process and .teacher performance needs to be more clearly 
delineated. If the instructional development process makes a difference, 
to whom does iti- matter? Are the dif^rential results •obtained from 
student groups attributable more to restructured course content? to 
varied instructional delivery systems? to enhanced faculty performance? 
or, as seems more l-ikely, interaction among these^ variables? Does 
the instructional development proces^s benefit students directly? Or 
are the benefits students derive mediated through the involvement of 
faculty rtiembers in the course development process, student benefits being, 
therefore, of a second and differervk order? 

The movement from ev.aluation df single courses to assessment of the 
tumulative effects of systema.tic course destgn on attitudes toward the 
total, academic program will require careful definition of what., 
constitutes "an academic program," ks well as to the delineation of the 
characteristics of such programs, jwhat are the differences between a - 
"curricijlum" and an "academic program"?^ Overall,, the seven systematically, 
designed courses used for grouping: purposes in this s'tudy stressed, to 
varying {degrees^ the following elements: 1) self-pacing; 2) flexible 
credit;| 3) aij^Jncreased number ofjcontent options (when compared with 
•nore^ conventional courses); 4) a variety"* of teaching strategies; 
5) use |of self-iristructional materials;, interdisciplinary content and 
7) a cqnsiderable amount of built-in formative evaluation. Which (if any) 
of these elements can be associated w^th the differential student 
perceptions of their academic program? :VJhat are the relative contributions 
of eacli? J \ ' ° t 

Ih m|ny. respepts, this study is. exploratory at best. It raises/al 
least las' many questions as it answers. - But it also tentatively esiablishe 
a 1.in| between freshman students', exposure to systematically designed 
courses and more positive attitudes toward their academic program. 
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